Content delivery networks (CDNs) enable provision of content, such as audio and video content, to users. The CDNs typically employ a multi-tiered hierarchy of computer servers, with edge cache servers that serve requests for content from client devices, intermediate or mid-tier cache servers that serve requests from edge cache servers and other mid-tier cache servers, and an origin server that originally stores and supplies the content. The purpose of this hierarchy is to create a scaleout network that can serve a large number of user requests. However, if only one origin server serves each user request, that server would quickly be overloaded if the content it served grew in popularity.
In a typical system, every user request goes through an edge cache server, regardless of the popularity of the content being requested. When the content library being served by the network is very large or has a “long tail,” it may become difficult to efficiently manage the edge cache servers. One conventional solution is to use page replacement algorithms, such as Least Recently Used (LRU) or LRU-K, to purge content from a cache server when it has not been recently requested.
When using a page replacement algorithm to purge unpopular content, the content is still served and cached by an edge cache server and any mid-tier cache servers. For example, if an origin server hosts a content library of 1,000 videos, and one of the videos is requested by only one user, the user request is served by the origin server, any mid-tier cache servers, and the edge cache server. If LRU is being used, then not only is the request served by multiple cache servers, but the content is also cached by the cache servers that request it. If a cache server is full, then caching new content requires the purge of already-cached content. No consideration is given to the fact that the purged content may actually be more popular than the newly-cached content.
With a sufficiently large content library this may lead to severe cache thrashing. A cache is considered to be thrashing when it is constantly purging and filling content, as opposed to serving from the cache. A severely thrashing cache may put as much load on the network as routing all requests to the origin. It may be very difficult to size an edge cache server appropriately for a large and growing content library.
Accordingly, there remains a need to improve the delivery of content, and to balance that need with the strains on the network.